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PREFACE 


This  manual  outlines  a  procedure  for  integrating  the  Quality 
Control  (QC)  samples  for  the  Medicaid,  AFDC  and  Food  Stamp 
QC  programs,    Such  integration  has  been  tested  out  in  several 
States  and  has  proven  to: 

.    Significantly  reduce  the  numbers  of  staff 
and  associated  costs  required  for  the  qc 

EFFORT. 


The  Medicaid  Bureau  hopes  States  will  find  this  manual 
useful,    Any  State  wishing  to  implement  an  integrated 
sampling  approach  or  having  any  questions  on  this  approach 
should  contact  its  Regional  Medicaid  Quality  Control  Office. 


Greatly  improve  the  utility  of  th  fdata 
collected  from  the  qc  review  process. 
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I.  INTRODUCTION 

A.    Rationale  for  Integrated  Sample  Selection 

There  are  three  federally  mandated  hut  state  administered  quality- 
control  systems  for  public  assistance  programs.    These  are  the  AFDC, 
Medicaid,  and  Food  Stamp  Quality  Control  programs.     The  purpose  of 
these  quality  control  systems  is  to  measure  and  reduce  the  frequency 
of  benefit  errors,  which  are  benefits  disbursed  for  ineligible  recipients 
or  incorrect  benefit  amounts  for  eligible  recipients. 

The  three  quality  control  systems  have  many  similarities.     Each  is 
based  on  a  monthly  random  sample  of  cases.     The  quality  control  sample 
cases  are  investigated  thoroughly  to  determine  if  errors  have  been  made 
in  eligibility  determination  or  the  benefit  amount  for  the  review  month. 
The  quality  control  review  also  collects  information  about  the  causes  of 
any  errors  found.    This  information  is  used  for  corrective  action  planning 
to  eliminate  the  causes  of  benefit  errors. 

In  the  past,  there  has  been  little  interaction  between  the  three 
quaLity  control  systems.     Each  system  had  its  own  sample  (unrelated  to 
the  other  system  sample:').    When  a  case  was  reviewed,  only  the  benefit, 
for  the  program  in  whose  sample  the  case  fell  was  reviewed  even  though 
the  same  case  might  have  also  been  receiving  benefits  from  another 
program.     In  most  states,  individual  quality  control  reviewers  conducted 
only  one  type  of  review  (AFDC,  Food  Stamp  or  Medicaid). 

Until  recently,  this  was  a  reasonable  approach,  since  there  was  not 
much  overlap  in  the  universes  sampled  for  the  three  quality  control  systems. 
Before  July  1976,  AFDC  cases  receiving  food  stamps  were  excluded  from  the 
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Food  Stamp  Quality  Control  system.     Thus,  there  was  no  overlap  in  the 
universes  for  the  AFDC  and  Food  Stamp  Quality  Control  systems.     The  Food 
Stamp  Quality  Control  system  has  now  been  expanded  to  include  AFDC  cases 
receiving  Food  Stamps.    The  Medicaid  Eligibility  Quality  Control  system 
did  not,  but  now  does  include  AFDC  cases  (as  well  as  some  other  types  of 
cases  which  were  previously  excluded,  e.g.  SSI  cases).    The  Medicaid 
Quality  Control  system  has  now  been  modified  to  use  a  sample  of  eligible 
cases  rather  than  a  sample  of  paid  claims.    Figure  1  shows  the  relationship 
which  exists  between  the  universes  for  the  three  quality  control  systems. 


Figure  1.     Overlapping  universes   for  AFDC , 

Food  Stamp,  and  Medicaid  quality  control 
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Many  families  receive  more  than  one  of  the  benefits  being  considered. 
There  would  be  a  considerable  degree  of  efficiency  involved  if  a  family 
could  be  included  in  more  than  one  quality  control  sample.    One  home  visit 
could  replace  two  or  three,  thereby  reducing  travel,  a  significant  cost 
in  quality  control.     Since  many  of  the  items  (e.g.  assets  and  income)  -which 
have  to  be  verified  pertain  to  all  three  programs,  and  for  other  reasons 
discussed  later,  the  cost  of  doing  two  reviews  on  one  case  is  significantly 
less  than  the  cost  of  doing  two  reviews  on  two  different  cases. 

The  term  integrated  sampling  has  been  adopted  to  describe  various 
methods  whereby  cases  selected  for  one  quality  control  sample  can  also  be 
utilized  as  part  of  one  (or  two)  of  the  other  quality  control  samples. 
Several  different  approaches  to  integrated  sampling  are  described  in 
this  guide.    The  difference  among  the  various  approaches  are  in: 

1)  The  conditions  required  for  implementation  (e.g.  some  approaches 
require  that  the  benefit  combination  in  each  case  must  be  known 
while  the  sample  is  being  selected,  while  other  approaches  do  not). 

2)  The  end  result  (some  approaches  produce  self-weighting  samples 
while  the  other  approaches  produce  stratified  samples  which  must 
be  combined  with  weighting  factors). 

3)  The  relative  efficiency  (some  approaches  are  less  efficient  because 
less  multiple  benefit  cases  are  selected  or  not  all  of  the  benefits 
are  reviewed  in  some  of  the  multiple  benefit  cases). 
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lt should  be  noted  that  there  are  economies  involved  in  having 
generalized,  or  generic,  quality  control  reviewers  (reviewers  who  review 
cases  in  more  than  one  program)  even  if  integrated  sample  selection  is 
not  used.     Since  generic  reviewers  can  perform  more  functions,  each  can 
be  assigned  to  a  smaller  geographical  area  which  reduces  the  total  time 
and  money  spent  travelling.    Also,  having  cases  to  be  reviewed  located 
closer  together  reduces  time  spent  waiting  if  time  that  would  otherwise 
be  spent  waiting  on  a  case  can  be  used  to  do  something  on  another  case. 
Thus,  a  significant  reduction  in  quality  control  costs  may  be  achieved 
by  using  generic  reviewers. 

For  the  full  potential  efficiencies  of  integrated  sampling  to  be 
realized,  quality  control  reviewers  must  be  able  to  complete  any  type 
of  quality  control  review.    The  extra  work  that  is  involved  in  doing 
more  than  one  review  in  some  cases  is  more  than  offset  by  the  economies 
involved  in  having  generic  workers.    Thus,  the  percentage  reduction  in 
quality  control  review  costs  that  can  be  achieved  with  generic  reviewers 
and  integrated  sample  selection  is  at  least  as  large  as  the  percentage 
reduction  in  the  number  of  distinct  cases  to  be  reviewed. 

With  some  of  the  integrated  sampling  procedures  described  later, 
the  number  of  distinct  cases  to  be  reviewed  can  be  reduced  by  about 
50$,  compared  to  the  number  of  cases  that  would  have  to  be  reviewed 
if  completely  independent  samples  were  used  for  AFDC,  Medicaid,  and 
Food  Stamp  Quality  Control. 

Since  50$>  of  the  overall  quality  control  active  case  review  costs  is 
a  significant  amount  of  money  in  a  state,  integrated  sample  selection 
should  be  considered  by  states  in  spite  of  the  additional  complications 
involved. 
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B.    History  of  Integrated  Quality  Control  Sample  Selection 

The  first  opportunity  and  incentive  for  integrated  quality  control 
sample  selection  occurred  in  1976  when  the  Food  Stamp  Quality  Control  system 
was  expanded  to  include  AFDC  Food  Stamp  households,  which  had  previously 
been  excluded  from  Food  Stamp  Quality  Control.     The  AFDC  Food  Stamp  house- 
holds were  the  first  substantial  group  of  cases  to  be  sampled  from  for 
two  quality  control  systems.    Another  effect  of  the  1976  changes  was 
increased  Food  Stamp  Quality  Control  sample  sizes  in  many  states.  In 
some  of  the  smaller  states,  the  Food  Stamp  Quality  Control  sample  size 
was  increased  by  as  much  as  50$.    At  about  the  same  time,  quality  control 
workloads  were  also  increased  by  other  changes,  such  as  increased  numbers 
of  negative  case  action  reviews. 

Because  of  the  increasing  quality  control  sample  sizes,  there  was 
considerable  interest  in  methods  to  reduce  quality  control  workloads. 
Many  states  submitted  proposals  to  integrate  sample  selection  for  AFDC 
and  Food  Stamp  Quality  Control.     Some  of  these  were  given  tentative  approval 
by  the  Department  of  Health,  Education  and  Welfare  and  the  Department  of 
Agriculture.    However,  many  questions  about  integrated  sampling  remained 
unanswered.     To  obtain  answers  to  these  questions,  five  States  using 
integrated  sample  selection  were  visited,  and  their  integrated 
sampling  procedures  studied.    An  Evaluation  of  Integrated  Sample  Selection 
for  AFDC  and  Food  Stamp  Quality  Control  is  a  report  on  those  visits. 
Some  of  the  findings  are  summarized  here. 


Training  specialized  quality  control  reviewers  to  do  other  types  of 
quality  control  reviews  is  not  as  difficult  as  had  been  anticipated.  It 
can  be  accomplished  with  approximately  one  week  of  training  and  some 
additional  supervision  for  the  following  several  months.    There  was  no 
indication  of  any  reduction  in  the  quality  of  either  type  of  review 
when  both  types  of  reviews  are  done  by  all  workers.    Also,  doing  more 
than  one  type  of  review  in  the  same  case  did  not  appear  to  reduce  the 
quality  of  any  of  the  reviews. 

The  only  negative  feedback  on  the  integrated  review  process  was 
concern  with  the  duplication  of  effort  involved  in  copying  much  of  the 
same  information  onto  the  separate  worksheets  for  each  review  in  a 
multiple  benefit  case.    An  integrated  worksheet,  to  collect  all  the 
information  needed  to  fill  out  any  of  the  quality  control  review  schedues, 
has  been  developed  and  is  being  tested.     If  it  proves  effective  it 
should  greatly  facilitate  the  integrated  review  process. 

One  of  the  concerns  about  integrated  sampling  is  that  the  completion 
of  reviews  might  be  delayed.     In  some  cases  they  are,  but  the  delays 
are  not  long  enough  to  cause  problems  in  getting  accurate  information 
or  meeting  reporting  deadlines. 

The  delays  which  do  occur  are  caused  by  waiting  for  the  Food  Stamp 
participation  lists  so  that  when  an  AFDC  or  Medicaid  case  is  assigned, 
it  can  be  known  whether  it  is  a  Food  Stamp  case  also.     The  best  way 
to  avoid  this  is  to  draw  the_JFooA_Sjtamp  Quality^  Control  sample  from 
authorization  lists,  which  are  available  during  the  review  month.  The 
reviews  can  be  initiated  promptly  and  cases  can  be  dropped  from  Food 
Stamp  Quality  Control  if  they  do  not  use  their  authorization  to  purchase 
food  stamps. 
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When  integrated  sample  selection  is  used,  there  is  a  strong  argument 
for  not  initiating  any  of  the  reviews  until  the  review  month  is  over. 
The  amount  of  Food  Stamps  and  Medicaid  "benefits  (if  any)  that  a  case 
uses  during  the  review  month  could  be  influenced  by  either  local  agency 
or  client  knowledge  of  quality  control  sample  selection  during  the 
review  month.    The  only  way  to  avoid  this  potential  source  of  bias 
in  the  quality  control  findings  would  be  to  select  the  quality  control 
samples  in  the  month  following  the  review  month.     (Because  of  this 
potential  problem,  this  procedure  may  yet  be  mandated  in  Medicaid  Quality 
Control  for  non-AFDC  cases). 

Integrated  sample  selection  is  more  complicated  than  independent 
sample  selection.    However,  if  the  complications  are  handled  properly, 
the  integrated  samples  are  as  statistically  valid  as  independent  samples. 
How  much  more  complicated  integrated  sample  selection  is  depends  on 
the  method  used  to  obtain  integrated  samples.    The  first  States  to  inte- 
grate quality  control  sample  selection  all  used  some  variation  of  what 
is  described  later  as  the  replacement  method.    The  complications  and 
frame  requirements  for  this  method  were  a  barrier  to  the  use  of  inte- 
grated sampling  in  some  states.    For  that  reason,  other  methods  have 
been  developed  with  less  complications  and  frame  requirements. 


C.     The  Expanded  Medicaid  Quality  Control  System 

The  expanded  Medicaid  Quality  Control  System  is  designed  to  provide 
much  more  useful  information  than  the  previous  MEQC  system,  which  used 
relatively  small  samples  compared  to  other  quality  control  systems  and 
covered  only  part  of  the  total  population  of  Medicaid  recipients.  The 
expanded  Medicaid  Quality  Control  sample  sizes  are  several  times  as 
large  as  the  previous  MEQC  sample  sizes  in  most  states.    However,  by 
integrating  the  Medicaid  Quality  Control  sample  selection  with  samples 
from  other  quality  control  systems,  this  expanded  system  can  be  implemented 
with  only  a  small  increase  or  even  a  decrease  in  the  total  quality 
control  workload  in  a  state. 

The  basic  sampling  design  for  the  expanded  Medicaid  Quality  Control 
system  is  a  stratified  sample.     In  States  which  have  a  163^-  contract 
with  the  Federal  Government  to  determine  eligibility  for  SSI  cases, 
there  are  three  strata  as  shown  in  figure  2. 


Figure  2.     Strata  for  MQC   in   1634  states. 


The  AFDC  and  SSI  strata  are  already  covered  by  other  quality  control 
systems  (AFDC  Quality  Control  and  SSI  Quality  Assurance).    The  MQC  sampling 
design  involves  using  all  of  the  AFDC  Quality  Control  and  SSI  Quality 
Assurance  sample  cases  as  part  of  the  Medicaid  Quality  Control  sample. 
The  rationale  for  this  is  that  AFDC  and  SSI  cases  are  automatically 
eligible  for  Medicaid.    Therefore,  a  separate 

field  investigation  is  not  necessary  to  determine  Medicaid  eligibility 
if  a  definitive  determination  of  AFDC  or  SSI  eligibility  has  been  made. 

The  number  of  Medicaid  Quality  Control  sample  cases  to  be  selected 
from  each  stratum  in  each  state  is  presented  in  the  MQC  Sampling  Manual. 
In  the  AFDC  and  SSI  strata,  the  number  of  MQC  sample  cases  is  determined 
by  the  AFDC  Quality  Control  and  SSI  Quality  Assurance  sample  sizes. 
The  number  of  cases  to  be  selected  from  the  Medical  Assistance  Only 
strata  is  the  same  number  of  cases  that  were  reviewed  in  the  previous 
MEQC  system. 

In  states  which  do  not  have  a  163^+  contract,  there  are  only  two 
separate  strata  in  the  MQC  sampling  design.    The  two  strata  are  l)  AFDC 
cases,  and  2)  all  other  Medicaid  cases.    The  SSI  cases,  which  are 
included  in  the  non-AFDC  stratum,  are  not  treated  as  a  separate  stratum 
because  the  state  determines  Medicaid  eligibility  for  SSI  cases  in  non-l63*+ 
states. 

The  number  of  MQC  sample  cases  to  be  selected  from  the  non-AFDC 
stratum  in  non-163^-  states  is  the  same  as  the  number  of  cases  reviewed 
in  the  previous  MEQC  system.     Those  SSI  cases  selected  as  part  of  the 
non-AFDC  stratum  may  be  replaced  by  cases  reviewed  by  SSI-GA. 


-10- 


D.    Additional  Integration  Possibilities 

The  basic  Medicaid  Quality  Control  sampling  design  uses  integrated 
sample  selection  to  some  extent,  but  not  to  the  full  extent  possible. 
Integrating  Medicaid  Quality  Control  sample  selection  with  the  sample 
selections  for  AFDC  Quality  Control  and  also  with  SSI  Quality  Assurance 
made  it  possible  to  enlarge  the  Medicaid  Quality  Control  sample  by  about 
four  times  (from  17,575  to  77>^10)  without  increasing  the  number  of 
quality  control  cases  with  field  investigations. 

By  taking  advantage  of  additional  integration  possibilities  among 
AFDC,  Food  Stamp,  and  Medicaid  Quality  Control,  a  state  can  implement 
the  expanded  Medicaid  Quality  Control  system  and  end  up  reviewing  less 
cases  than  are  currently  being  reviewed. 

Figure  3  is  a  schematic  representation  of  the  quality  control  sample 
selections  assumed  in  the  basic  MQC  sampling  design. 


Food  Stamp  QC        AFDC  QC  SSI  QA 


Medicaid  QC      Medicaid  QC     Medicaid  QC 


Figure  3. 


Cases  included 


in  various  QC 


sys  t  ems . 
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The  additional  integration  that  is  possible  is  shown  in  figure  k. 

o  ^ 

I'ii'.uro   i.     Acltl  1  f  i  on.i  1    i  ntcgrnl:  ion  poss  i  l>  i  I  i  t  i  cs  . 

Essentially,  Food  Stamp  QC  can  now  be  integrated  with  both  AFDC-QC 
and  the  Medical  Assistance  (non-AFDC)  part  of  Medicaid  QC. 

In  l63^  states,  there  would  be  no  advantage  to  integrating  the  SSI 
Quality  Assurance  cases  with  any  of  the  quality  control  systems  except 
Medicaid,  since  states  do  not  have  to  do  field  investigations  in  these 
cases . 

There  are  many  different  methods  which  can  be  used  to  obtain  integrated 
samples  from  overlapping  universes.     Each  method  has  its  advantages  and 
disadvantages.    The  process  of  selecting  the  integrated  sampling  method 
to  use  would  involve,  first  of  all,  eliminating  those  which  could  not  be 
implemented  because  of  some  barrier  such  as  the  lack  of  required  sample 
frames.     Then,  if  more  than  one  method  could  be  implemented,  priorities 
must  be  determined  for  the  other  features.     For  example,  which  is  more 
important,  efficiency  or  self -weighting  samples. 

The  following  sections  describe  several  methods  for  selecting  integrated 
samples,  examples  of  each,  and  the  sampling  frame  and  weighting  requirements. 
The  information  presented  should  enable  a  state  to  understand  each  method. 
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II.     REPLACEMENT  METHODS  OF  SELECTING  INTEGRATED  SAMPLES 
A.  Introduction 

Replacement  methods  of  selecting  integrated  samples  use  stratified 
samples  from  overlapping  universes.     Consider,  for  example,  the  two 
overlapping  universes  in  figure  5. 


IMp urc  5.      Two  overlapping  Figure  0.     Stratified  sample 

universes.  from  each  universe. 

Each  of  the  two  universes  consists  of  two  strata,  the  overlapping 
part  (ME)  and  the  non-overlapping  part  (M  or  F) .    A  stratified  sample 
is  obtained  from  each  of  the  two  universes,  as  shown  in  figure  6. 
One  of  the  two  samples  from  the  overlapping  stratum  MF  is  deleted  (or 
not  even  selected)  and  is  replaced  by  some  or  all  of  the  cases  in  the 
other  sample  from  stratum  MF.     If  the  deleted  sample  is  replaced  by- 
all  of  the  other  sample  from  the  overlapping  stratum,  then  one  of  the 
resulting  stratified  samples  (F  +  MF  in  this  example)  is  not  a  propor- 
tional stratified  sample  -  which  means  that  weighting  must  be  used 
in  the  analysis  of  that  sample. 

Self-weighting  samples  for  both  universes  can  be  obtained  by 
replacing  the  smaller  sample  from  MF  with  a  random  subsample  of  the 
same  size  from  the  other  sample  from  MF. 
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The  basic  Medicaid  Quality  Control  sampling  design  can  be  viewed 
as  the  non-proportional  replacement  method.     In  163^  states,  the  Medicaid 
eligibility  file  is  stratified  as  shown  in  figure  7.     If  a  sample  were 
taken  from  the  entire  Medicaid  eligibility  file,  there  would  be  some 
AFDC  cases,  some  SSI  cases,  and  some  other  Medicaid  cases.  Integrated 
sample  selection  is  accomplished  by  replacing  the  AFDC  cases  with  the 
cases  from  the  AFDC  Quality  Control  sample  and  replacing  the  SSI  cases 
with  cases  from  the  SSI  Quality  Assurance  sample. 


Figure  7.     Strata  for  MQC  sample  selection  in  1634  states. 

In  non-163^-  states,  there  are  only  two  strata  for  MQC  sample  selection. 
These  are  l)  AFDC  cases  and  2)  all  other  Medicaid  cases.     Sample  selection 
is  integrated  with  the  AFDC  Quality  Control  sample,  by  using  the  entire 
AFDC  Quality  Control  sample  in  place  of  the  AFDC  cases  that  would  otherwise 
have  been  obtained  by  sampling  from  the  entire  Medicaid  eligibility 
file,  and  with  SSI  by  substituting,  for  SSI  cases  falling  into  the 
sample,  SSI-QA  cases  determined  eligibility  for  Medicaid. 

The  replacement  method  can  also  be  used  to  obtain  integrated  samples 
for  all  three  state  operated  quality  control  systems  (AFDC,  Food  Stamps, 
and  Medicaid).     The  overlapping  universes  for  these  three  quality  control 
systems  are  shown  in  figure  8. 


I'igurc  8.     Overlapping  universes   for  AJ;I)C,   Food  Stamp,  and 
Medicaid  quality  control. 

The  Medicaid  universe  to  be  sampled  from  consists  of  four  strata, 
or  types  of  cases.     The  two  non-AFDC  strata  (M  &  MF)  contain  the  types 
of  non-AFDC  cases  which  are  to  be  selected  for  field  investigations. 
In  163^  states,  SSI  cases  are  excluded.     In  non-163^-  states,  SSI  cases 
are  included  in  the  two  non-AFDC  strata  M  &  MF). 

The  replacement  method  sampling  procedure  for  integrating  the  Medicaid, 
AFDC,  and  Food  Stamp  Quality  Control  is  presented  in  the  next  section. 
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B.    Sampling  Procedure 

1.  Select  the  AFDC  Quality  Control  sample  in  the  normal  manner 
using  the  interval 

I    =  estimated  average  caseload  size  g 
required  AFDC  sample  size 
(allowing  for  drops) 

All  of  these  cases  will  be  included  in  the  MQC  sample.  All 

(or  part)  of  the  cases  which  receive  Food  Stamps  will  be 

included  in  the  Food  Stamp  Quality  Control  sample. 

2.  Select  the  sample  of  non-AFDC  Medicaid  cases*  to  be  reviewed 
using  the  interval 

estimated  average  caseload  size* 
Im  =  required  non-AFDC  MQC  sample  size  X° 
(allowing  for  drops) 

*This  is  the  number  of  Medical  Assistance  Only  cases  in  163^-  states 
and  the  number  of  non-AFDC  Medicaid  cases  (including  SSI  cases) 
in  non-163^-  states.     Some  or  all  of  these  will  be  included  in 
the  Food  Stamp  Quality  Control  sample. 

3.  Select  a  sample  of  Food  Stamp  only  cases  (excluding  any  Food 
Stamp  cases  which  are  also  receiving  AFDC  or  are  Medicaid  bene- 
ficiaries) using  the  interval 

estimated  average  Food  Stamp  only  caseload  size  g 
f  ~  required  Food  Stamp  only  sample  size 

(allowing  for  drops) 

The  size  of  the  Food  Stamp  only  sample  is  the  required  Food  Stamp 

sample  size  minus  the  number  of  Food  Stamp  cases  obtained  in 

steps  1  &  2. 
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C .  Example 

This  example  illustrates  the  use  of  the  replacement  method  in  a 
163k  state  with  caseload  configuration  shown  in  figure  ^.     SSI  cases 
have  been  excluded  from  the  Medicaid  universe. 


Food  Stamp 
Caseload  (total 

Non-cash  Medicaid 
Caseload  (total) 

AFDC  Caseload 


Caseload 


55,000 


8,000 


21,000 


Required 
Sample 

1,200 


175 
550 


Figure  9.     Overlapping  universes  with  SSI  cases  not 
included  in  the  Medicaid  universe. 


The  Food  Stamp  sampling  interval  will  be 


55,000 


f  (l.l)(1200) 


x  6  =  la. 67  x  6  =  250 


allowing  for  a 
10$  drop  rate 


The  number  of  Food  Stamp  cases  selected  for  a  six  month  period 

will  be  about  1320.  Of  these  about 

1320  X    2400    =  58 
55,000 

will  be  from  strata  MF.     These  will  be  replaced  by  that  number  (58)  of 

MF  cases  obtained  while  selecting  the  Medicaid  Quality  Control  sample. 

There  will  be  about  1320  X    IgalSg  =  353  amf  cases  in  the  Food 

55,000 

Stamp  sample.     These  will  be  replaced  by  that  number  (353)  of  AMF  cases 
obtained  while  selecting  the  AFDC  sample. 
The  AFDC  interval  will  be 


21,000  21,000 
Za  =  (l.l5(550)  X  6=T05~  X  6  =  208 


The  605  AFDC  sample  cases  will  provide  about 

605  X  14,700 


r allowing  for  a 
lCffo  drop  rate 


21,000 


=  423  AMF  cases. 
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To  obtain  a  self --weighting  Food  Stamp  sample,  only  a  random  sample 
of  353  of  these  should  be  selected  for  Food  Stamp  reviews. 
The  Medicaid  interval  will  be 

r«  =  n  ?w?7^  X  6  =  249  fallowing  for  a 

U-LiU/3J  drop  rate 

Of  the  192  cases  that  will  be  selected,  about  192  X  2^00  =  58 

©000 

will  be  MF  cases.    This  is  exactly  the  number  needed  to  replace  the 
58  MF  cases  in  the  Food  Stamp  sample. 

The  table  below  shows  the  number  of  cases  and  reviews  for  each 
benefit  combination.    The  total  number  of  quality  control  reviews  is 
2922.    There  are  only  1706  cases  selected  for  field  investigation  which 
is  a  kzfo  reduction  due  to  integrated  sample  selection. 


Benefit 

Cases 

AFDC 

Medicaid 

Food  Stamp 

Combination 

Selected 

Reviews 

Reviews 

Reviews 

F 

909 

909 

MF 

58 

58 

58 

AMF 

*+23 

h23 

h23 

353 

AM 

182 

182 

182 

M 

13^ 

131+ 

1706 

605 

797 

1320 

+200  SSI 

997 

In  this  example,  disproportionate  replacement  would  have  resulted  in 
70  less  Food  Stamp  only  cases  to  be  reviewed.    This  additional  2%  reduc- 
tion in  cases  to  be  reviewed  would  probably  not  be  worth  the  loss  of 
a  self -weighting  Food  Stamp  Quality  Control  sample.     In  other  caseload 
configurations  disproportionate  replacement  might  be  considerably  more 
efficient. 
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D.    Frame  Requirements 

For  this  method  to  be  implemented,  the  following  conditions  should 
he  met. 

1)  It  must  he  possible  to  identify  (at  the  time  of  sample  selection) 
those  AFDC  Quality  Control  sample  cases  "which  also  receive  Food 
Stamps. 

2)  The  fraome  used  to  select  the  Food  Stamp  Quality  Control  sample 
cases  must  specify  which  are  AFDC  cases  and  which  are  non-AFDC 
cases  and  which  are  non-AFDC  Medicaid  cases. 

3)  The  frame  used  to  select  Medicaid  cases  must  specify  which  cases 
are  AFDC,  and  in  163^-  states,  which  cases  are  SSI  and  which  cases 
are  neither  of  these.     It  must  also  specify  which  cases  receive 
Food  Stamps. 
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Weighting  Required 

This  method  results  in  the  following  -weighting  requirements. 

No  weighting  is  required  in  the  compilation  of  AFDC  Quality  Control 

findings . 

Weighting  is  always  required  in  the  compilation  of  Medicaid  Quality 
Control  findings  and  is  required  for  Food  Stamp  Quality  Control 
if  disproportionate  replacement  is  used.    Weighting  methods  are 
explained  in  Part  E  of  Section  IV. 
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III.     GOCIAL  SECURITY  NUMBER  METHOD 
A.  Introduction 

This  method  was  first  used  by  the  South  Dakota  Department  of  Social 
Services  to  integrate  sample  selection  for  AFDC  and  Food  Stamp  Quality- 
Control.    This  method  is  of  special  interest  because  it  produces  self- 
weighting  samples  without  requiring  that  the  benefit  combination  of  cases 
be  indicated  in  the  sampling  frames.    This  method  assumes  that  the  last 
four  digits  of  the  Social  Security  numbers  in  a  given  state  caseload 
are  random. 
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B.     Sampling  Procedure 

Each  month  some  method  is  used  to  generate  a  pre-specified  number 
of  four  digit  random  numbers.    The  number  of  random  numbers  needed 
to  select  the  AFDC  Quality  Control  sample  may  be  different  from  the 
number  of  random  numbers  needed  to  select  the  Food 

Stamp  sample.    Let  Ra    R^.,  and  R^  be  the  respective  numbers  of  random 
numbers  used  for  the  three  samples. 

The  AFDC  Quality  Control  sample  is  obtained  by  comparing  the 
last  four  digits  of  the  Social  Security  number  of  the  payee  in  each 
AFDC  case  to  all  RQ  random  numbers  being  used  to  select  the  AFDC 

EL 

Quality  Control  sample.    If  there  is  a  match,  the  case  is  selected 
to  be  included  in  the  AFDC  Quality  Control  sample.     The  probability 

of  selection  for  any  AFDC  case  is  Rg/10,000.    Ra  is  determined  as  follows. 

R  Average  Monthly  Sample  Size  1Q  00Q 

a  "  Estimated  Average  Caseload  Size  ' 


If  the  estimated  average  AFDC  caseload  size  is  21,000  and  the  average 
monthly  sample  size  is  550/6  =  92,  then 

_    92  X  10,000  =  kk 
na  21,000 

The  Food  Stamp  Quality  control  sample  is  selected  in  the  same  way, 
using  the  Social  Security  number  of  the  Food  Stamp  household  head. 
R_,  the  number  of  random  numbers  needed  to  select  the  Food  Stamp  sample, 
is 

Average  Monthly  Sample  Size      x  1Q  00Q 
f  ~    Estimated  Average  Caseload  Size  ' 

If  the  estimated  average  Food  Stamp  caseload  is  55,000  and  the 

average  monthly  sample  size  needed  is  200,  then 

200  X  10,000  _  q7 
Rf  ~         55,000       ~  i( 


-22- 


To  obtain  an  integrated  sample,  37  of  the  same  random  numbers  used 
to  select  the  AFDC  sample  must  be  used. 

The  Medicaid  Quality  Control  sample  is  selected  in  the  same  way, 
using  the  Social  Security  number  of  the  primary  person  in  the  Medicaid 
case.    Rm,  the  number  of  random  numbers  needed  to  select  the  non-AFDC 
Medicaid  sample  is 

_         Average  Monthly  Sample  Size       x  1Q  Q00 
«  ~      Estimated  Average  Caseload  Size  ' 

If  the  estimated  average  caseload  size  is  8,000  and  the  average 

monthly  non-AFDC  Medicaid  sample  size  is  175/6,  then 

=    (175/6)  X  10,000  =  6 
\  8,000  ~  JD 

To  obtain  an  integrated  sample,  36  of  the  random  numbers  used  to 
select  AFDC  and  Food  Stamp  cases  are  used  to  select  the  Medicaid  sample. 
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C .  Example 

The  table 

below  shows  the  number  of  cases 

of  each  type 

in  the  quality 

control  samples  in  the  example  state. 

The  total 

number  of  cases  with  field  investigations  is 

1571,  which 

is  k2^o  less  than  the  total  number  of  reviews 

f  2704) . 

Benefit 
Combination 

Caseload  AFDC 
Size             Sample  Cases 

Medicaid 
Sample  Cases 

Food  Stamp 
Sample  Cases 

AMF 

lU,700  388 

388 

326 

AM 

6,300  166 

loo 

MF 

2,k00 

53 

53 

M 

5,600 

123 

P 

37,900   

840 

SSI 

55^ 

730 
200 
930 

1220 
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D.    Frame  Requirements 

The  only  frame  requirement  for  this  method  is  that  the  frames  for 
selecting  the  three  quality  control  samples  all  contain  a  Social 
Security  number  for  each  case.    The  statistical  validity  of  this 
method  is  not  affected  by  an  occasional  case  with  no  Social  Security 
number  if  a  provision  is  made  to  generate  random  numbers  for  the 
missing  Social  Security  numbers.     This  "will,  however,  reduce  the  amount 
of  overlap  in  the  samples  and  thus  reduce  the  efficiency  of  the  method. 
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E.    Weighting  Required 

The  Social  Security  number  method  produces  self -weighting  samples 
for  the  AFDC  and  Food  Stamp  Quality  Control  systems  and  the  normal 
stratified  sample  for  the  Medicaid  Quality  Control  system. 
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IV.     ADDITION  METHOD  OF  SELECTING  INTEGRATED  SAMPLES 
A.  Introduction 

The  addition  method  of  selecting  integrated  samples  from  two  overlapping 
universes  is  shown  in  figure  10. 


The  sampling  procedure  is  to  obtain  a  simple  (or  systematic)  random 
sample  from  each  of  the  two  universes  and  to  review  all  of  the  benefits 
in  all  of  the  cases  selected.     This  is  called  the  addition  method  because 
the  cases  in  the  second  sample  from  the  overlap  domain  are  added  to 
(rather  than  being  replaced  by)  the  cases  in  the  first  sample  from  the 
overlap  domain. 

The  primary  advantage  of  this  method  is  that  each  universe  can  be 
sampled  from  independently.     That  is,  it  is  not  necessary  to  know  the 
benefit  combination  at  the  time  of  sample  selection.    Another  advantage 
of  this  method  is  that  a  greater  proportion  of  the  sample  cases  will 
be  multiple  benefit  cases  and  the  total  number  of  distinct  cases  will 
be  less  than  would  be  obtained  with  the  replacement  method.     The  primary 
disadvantage  of  this  method  is  that  data  analysis  and  reporting  for 
all  of  the  resulting  samples  will  require  -weighting. 


Addition  method  of  obtaining  integrated 

s  ;i  m  p  1  c  s  . 
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B.    Sampling  Intervals 

If  the  total  number  of  cases  to  be  reviewed  for  each  quality- 
control  system  is  a  specified  number,  the  sampling  intervals  to  be 
used  for  the  additional  method  of  obtaining  integrated  samples  are 
determined  as  follows.    The  overlapping  universes  for  AFDC,  Food 
Stamp,  and  Medicaid  Quality  Control  are  shown  in  Figure  11.    As  before, 
SSI  cases  are  excluded  from  the  non-AFDC  Medicaid  universe  in  163^ 
states  and  included  in  other  states. 


Figure  11 


Overlapping  universes  for  Medicaid,  AFDC, 
and  Food  Stamp  Quality  Control  sample 
selection . 

Applying  intervals  I  ,  1^,  and  Ia  to  their  respective  universes 
will  produce  samples  of  sizes  i^,  G^.,  and  Qa#     Let  Sm  be  the  required 
number  of  Medicaid  cases  to  be  selected  (excluding  AFDC  cases  in  all 
states,  and  also  excluding  SSI  cases  in  163^-  states).    Let  Sa  and  Sf 
be  the  required  numbers  of  cases  to  be  selected  for  AFDC  and  Food  Stamp 
quality  control  and       be  the  number  of  cases  in  stratum  j .  Then 

Sm  =  Qm  +  Qf   

%f  +  % 

=  Qm  +  Qf  P-ftQ  where  Pfm  =  Nmf 

Nmf  +  Nm 


Sa  =  %  +  Of  ^M. 


Namf  +  %  +  Ikf 

=  Qa  +  Qf  Paf  where  P&f  =  Namf  

N  N  Namf  +  Nf  +  Nmf 

Sf  =  Qf  +  Qa      amf  +  Qm  mf 

Namf  +  %f  +  \ 

=  Qf  +  Qa  Pfa  +  Qm  Pfm  where  Pfa  =  Namf 


Namf  +  Nam 


-28- 


Solving  these  three  equatioms  for  the  three  unknowns  (Q^,  Q  ,  Qf) 


Ciives 


%  = 

o 

HI 

Pmf 

Qa  = 

S 

a. 

"  % 

Paf 

Of  = 

sf 

-  S 

a 

Pfa 

1  - 

paf 

pfa 

The  sampling  intervals  are  then  computed  by  dividing  each  universe 
size  by  the  number  of  cases  to  be  selected. 

Im  =  %  +         x  6 
Qm 

ra  =  Nam  +  Wamf  x  6 

I_  =*  N    c  +  N  r  +  Nr 
f        ami        mf        f      x  6 

States  should  oversample  to  some  degree  (i.e.  reduce  the  sampling 
interval)   to  insure  that  minimum  sample  size  requirements  are  met  in 
all  programs. 


M4«       t.  l>i,.^4-\JU, 

"Wit  -  ' 
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C .  Example 

The  example  in  the  previous  section  assumed  the  following  caseload 
configuration  and  required  sample  sizes. 

.      Nm       =    2,1+00  Sm=  192 

Hmf     =  5,600 

Namf    =  lk>700  Sa  =  6°5 

Nam     =  6>300 

Nf       =  37,900  Sf  =1,320 

If  the  addition  method  were  used  in  this  same  situation  the  followin 

results  would  be  obtained.     The  number  of  cases  selected  from  each  of 

the  frames  would_be 

Of  =  Sf  -  Sa  Pfa  -  Sm  Pffr 
1  "  Paf  Pfa  "  Pmf  pfm 

=  1320  -  605  (.70)  -  192 (.30)    =  1052 
1  -  (.27M.70)  -  (.<M)(.30) 

^  =  Sm  "  Qf  Pmf  =  ^  "  1052(.(M)  =  M 
Qa  =  Sa  '  Qf  Paf  =  6°5  "  1052(-27)    =  321 
The  number  of  sample  cases  with  each  benefit  combination  would  be 
Mf      =  722  Mam    =    9&  Mamf  =  508 

=    90  Mta     »  102 


There  would  thus  be  1320  cases  selected  for  Food  Stamp  reviews, 

605  cases  selected  for  AFDC  reviews,  and  192  cases  (in  addition  to  the 

605  AFDC  cases)  selected  for  Medicaid  reviews.     The  total  number  of 

reviews  is  1320  +  605  +  605  +  192  +  200  =  2922.     The  number  of  distinct 

cases  with  field  investigations  is  722  +  90  +  508  +  96  +  102  =  1518. 

The  percentage  reduction  in  cases  with  field  investigations  is 

2922  -  1518 
2922 
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D.    Frame  Requirements 

This  is  the  only  method  which  does  not  have  any  sample  frame 
requirements  in  addition  to  the  frames  that  would  be  required  to  select, 
independent  samples  for  each  of  the  quality  control  systems.  However, 
estimates  of  the  percentage  of  multiple  benefit  cases  are  necessary. 
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E.    Weighting  Required 

Weighting  is  required  for  all  three  quality  control  samples. 

For  all  of  the  methods  of  selecting  integrated  samples  described 
here,  the  following  procedure  can  be  used  -whenever  -weighting  is  required. 
The  formula  for  computing  any  error  rate  is 


4 


Where  is  the  sum    over  all  strata  covered  by  the  QC  system  that 

h  the  error  rate  is  for. 

W,   =  Size  of  caseload  in  stratum  h 
h 


QC  system  universe  size 

h 


P,  is  the  error  rate  for  all  QC  sample  cases  from  stratum  h 


Consider,  for  example,  the  follo-wing  caseload  configura.tion  and 
error  rates. 


Benefit 
Combination 

Cases  in 
Universe 

Cases  in 
Sample 

Error 
Rate 

F 

37,900 

909 

20$ 

MF 

2,U00 

58 

22$ 

AMF 

lU,700 

k23 

12$ 

AM 

6,300 

182 

10$ 

M 

5,600 

13k 

M 

SSI 

1+2,000 

200 

2$ 

The  error  rates  for  AFDC,  Food  Stamp,  and  Medicaid  quality 
control  would  be 

P    =    iV00  X  12  +  6,3Uu  x  10  =  11.4% 

a        01   OOO   xj-.h/o 

^±,uuu  21,000 

Pf  =  37,900    x  20  +  X  22  +  J-VOO  X  12  =  17.95$ 

55,000  55,000  55,000 

=    2,1+00  x  22  +  IMOO  x  12  +    6,300      1Q        5,600        .       1+2,000  _  g 

^m      71,000  X  dd  +  71,000  X  Ld  +  71,000  x  10  +  71,000  X  14  +  71,000  X  2  ~  b' 
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V.     OPTIMUM  ALLOCATION  METHOD 
A.  Introduction 

The  previous  methods  all  result  in  the  selection  of  different  numbers 
of  cases  with  each  "benefit  combination.     None  of  the  allocations  are 
optimal  in  the  sense  of  minimizing  the  variance  of  error  rate  estimates. 
The  optimal  number  of  cases  with  a  particular  benefit  combination  to 
be  selected  can  be  computed  once  it  is  decided  what  to  minimize  and 
what  the  constraints  are.    An  example  of  an  optimum  allocation  of  cases 
based  on  one  set  of  assumptions  is  described  in  Appendix  A.     Note  that 
constraint  equations  for  minimum  sample  sizes  would  have  to  be  included 
if  this  method  was  used. 
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B.     Sampling  Procedure 

For  an  optimal  allocation  design,  each  case  would  ideally  be 
exposed  to  sampling  only  once  and  would  be  selected  with  a  probability 
which  would  be  the  same  for  all  cases  with  the  same  benefit  combination 
and  different  for  cases  with  different  benefit  combinations. 
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C.    Frame  Requirements 

To  select  cases  in  the  most  optimal  manner,  it  would  be  necessary 
to  have  a  single  frame  (computer  file  or  listing)  of  the  combined 
AFDC,  Food  Stamp  and  Medicaid  caseloads.     Each  case  should  be  listed 
only  once  and  the  benefit  combination  should  be  indicated.  Unless 
a  computer  file  of  this  type  already  exists,  it  would  probably  not 
be  worth  the  effort  to  create  such  a  frame  just  for  the  purpose  of 
integrated  sample  selection. 
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D.    Weighting  Requirements 

Pny  optimized  sample  will  require  -weighting  for  reporting  and 
data  analysis  in  all  three  quality  control  samples. 


E.     Relative  Efficiency 

The  measure  of  efficiency  that  has  been  used  here  to  compare  the 
various  methods  of  obtaining  integrated  samples  is  the  number  of  cases 
reviewed  divided  by  the  number  of  reviews  obtained.     Optimum  allocation 
may  or  may  not  be  more  efficient  in  this  respect  than  the  addition  method. 
The  efficiency  of  the  optimum  allocation  method  will  depend  on  the  con- 
straints that  are  set  and  the  variable  chosen  (variance,  cost,  sample 
size)  for  optimization.    Allocation  methods  can  be  deveoped  to  minimize 
total  reviews  while  meeting  minimum  requirements. 

The  table  below  shows  the  number  of  cases  with  each  benefit  combination 
that  would  be  selected  in  the  example  state  according  to  the  optimal 
allocation  in  Appendix  A,  an  allocation  which  minimizes  costs  subject 
to  keeping  variances  at  or  below  current  program  requirements  without 
regard  to  minimum  sample  sizes.     The  total  number  of  reviews  is  2960. 
There  is  a         reduction  in  the  number  of  cases  to  be  reviewed  with  this 
method  of  integrated  sample  selection. 


Benefit 
Combination 

Cases 
Selected 

AFDC 
Reviews 

Medicaid 
Reviews 

Food  Stamp 
Reviews 

F 

57^ 

MF 

262 

262 

262 

AMF 

koi 

koi 

1+01 

401 

AM 

155 

155 

155 

M 

3^8 

3kQ 

TOTAL 

17^0 

556 

1166 

1238 

APPENDIX  A 


An  Example  of  Optimal  Integrated  Samples 
Rudolph  E.  Schwartz,  Office  of  Research  and  Statistics 
Social  Security  Administration,  Woodlawn,  MD.  21235 


An  optimal  sample  takes  into  consideration  three  elements: 

(1)  Reliability  or  allowable  sampling  error. 

(2)  Variability  of  the  universe  or  sub-universes  from  which  samples 
are  drawn. 

(3)  Cost  of  sampling  which  considers  setup,  travel,  interview  and 
verification  costs. 

The  difficulty  with  designing  an  optimal  sample  is  that  each  of  the 
various  programs  has  its  own  values  for  the  three  elements  mentioned 
above.     Thus,  the  sample  that  is  optimal  for  one  program  may  not  be 
optimal  for  another. 

In  order  to  solve  such  a  problem  for  stratified  samples  and  several 
programs  one  must  solve  a  nonlinear  programming  problem. 

Mathematical  Formulation  of  an  Optimal  Stratified  Integrated  Sampling 
Problem  Samples 

MIN  C  =  2-a^r^  h  =  1  .   .   .  L,  0=1.   .   .  K 

Subject  to 

L  Wh2S23 

V.  ^K.  =21     ""^T         (neglecting  finite  correction  factor) 
J       3    h=l  n 

3=1.   .   .  K,  0  <       <  Nh 

Total  units  in  population 
Total  units  in  h  stratum 
Sample  units  in  nth  stratum 
N^/N  =  Stratum  weight 

Estimate  of  population  variance  in  h  stratum 

The  next  page  gives  an  example  of  a  typical  optimal  solution  which  has 
been  solved  by  a  nonlinear  program  package.     Since  many  of  the  States 
do  not  have  access  to  such  a  package,  we  will  be-happy  to  solve  a  Staters' 
design  problem  if  they  will  furnish  us  with  the  required  data. 


where  N 

Nh 
nh 

Sh2 


Example  —  Design  n-p 

«~  programs^ve  ^iSSiif^^'^^.*.  — 

 Program  c 

 Sample  SRS  Variance*  (10"U) 

Aid  for  Dependent  Children  (AFDC)  ft™ 

Food  Stamps  (FS)                           '                800  1.10 

Medicaid  (MED)                                             1,200  0.6l 

275  2.76 

l^T:.^xe:^tTe  variance  *•  ^ 

Stratum  r>      i  n. 

No  Population 

—  Size  (000)  Factor 

1  AFDC  9 

2  AFDC,  FS  18  \, 

3  AFDC,  FS,  MED  2  iVo 


AFDC,  MED 
FS 

FS,  MED 

7        MED  45  ,/2 


l       FS  23  i/2 

FS,  MED 


1 


Cost 

Unit 

Sampling 

Cost 

9.50 

9.50 

11.50 

5.75 

13.50 

4.50 

11.50 

5.75 

9.50 

9-50 

11.50 

5.75 

9.50 

9-50 

MINIMIZE: 

C  -  9.50^  +  n.50„2  +  13.50a3  +  n.50n4  +  9.50„5  +  XI. 50-    +  ,.50n 
SUBJECT  TO:  7 

0  <  \  <  j|  .  .  ^  o  .<  n7.<  N7  V  1 

<i§-!-»-  <M>'^.  ^^1^ 

J  n5  50  n6  xlO-4 

<3f)2  2^  +  ^  +  (^  +      »„>»->»>  <, 76 

*  °«  55  »7  *10-4 

*u  -  09,  PM  -  .10>  P31  .  08.  ,u  .  07 

II'  >  '  °9'        "  -10'  P«  ■  08  I 
Cost  -  17852  HRS 
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